Summary

A focused demonstration of SingleRust's performance and its in-place processing
ergonomics:

• examples/inplace_pipeline.rs — a minimal, annotated pipeline that loads an AnnData once and
  mutates it through every step (no copies).
• demo/scverse_benchmark.ipynb + examples/bench_step.rs — a per-step runtime benchmark vs
  the equivalent scanpy function on a ~50k-cell CZI CELLxGENE dataset.
• demo/scverse_scaling.ipynb — a cells-vs-runtime scaling sweep (3k → 50k).

Stacked PR: base is integration/features-only (write_h5ad, run_pca_inplace, dense
normalize/log1p, ORA, QC fix), so the diff here is just the benchmark, scaling, the in-place
example, and supporting scaffolding.

Why it's fast: in-place operations

Interior mutability means you pass a shared &adata to each step and results accumulate on the
same object — no copy between steps, the expression matrix is never reallocated.

let adata = io::read_h5ad_memory("input.h5ad")?;          // load ONCE — note: not `mut`
qc_metrics(&adata)?;                                       // -> obs / var
normalize_expression(&adata.x(), 10_000, &Direction::ROW, None)?;  // -> X in place
log1p_expression(&adata.x(), None)?;                       // -> X in place
compute_highly_variable_genes(&adata, Some(HVGParams { n_top_genes: Some(2000), ..Default::default() }))?;
run_pca_inplace::<f64>(&adata, Some(hvg_selection), Some(true), Some(false),
                       Some(50), None, Some(42), Some(svd), None)?;   // -> obsm["X_pca"], uns

Benchmark (per step, 50k × 35.5k, 18 cores)

Compute time only (Rust .h5ad read/write excluded); a warm-up pass runs first so scanpy isn't
charged for one-time numba JIT / thread-pool startup.

SingleRust wins decisively on the heavy/vectorizable steps (PCA ~6×, normalize ~8×). QC is the
honest exception — scanpy's optimized C path slightly beats SingleRust's qc_metrics (the top-N
segment proportions are the cost). * ORA has no scanpy equivalent; compared against a
NumPy/SciPy implementation of the same algorithm.

Scaling (runtime vs cells)

Faster at every size (3–9×). SingleRust scales ~linearly with non-zeros; scanpy carries higher
fixed per-step overhead, so the margin is largest at moderate sizes and narrows by 50k as both
become compute-bound.

Notes

• Timings are compute-only, single-run, machine-dependent — indicative, not micro-benchmarked.
• Both sides use all cores (Rust via rayon, scanpy via BLAS).
• SingleRust writes blosc/zstd-compressed .h5ad; reading from Python needs import hdf5plugin.

🤖 Generated with Claude Code

Scaling to 500k cells + memory (added)

demo/scverse_scaling_large.ipynb pushes to 500,000 cells (broadened all-assays blood query,
genes fixed across sizes) and tracks peak RSS alongside time — both tools run under
/usr/bin/time -l on identical input. Clean-state reference (48 GB / 18-core, large sizes
measured in isolation):

• Scaling holds, speedup converges. SingleRust is faster at every size, but the pure-compute
  margin narrows from ~5.8× (25k) to ~1.6× (500k) — small-N wins are low fixed overhead; at
  scale both are compute-bound (PCA dominates), gap ~1.6× (normalize/HVG/PCA 2–4×, QC at parity).
• Memory is a confound, and it favors SingleRust. Peak RSS is ~2× smaller (≈10 vs ≈18 GB
  at 500k), so it stays compute-bound where scanpy starts paging. The notebook explicitly flags
  the two large-N confounds on a laptop — memory pressure (inflates scanpy 2–4× near the RAM
  limit while RSS plateaus) and thermal throttling under sustained benchmarking (slows both). RSS
  is immune to both, so the lower-footprint result is the robust takeaway.